Image processing apparatus, ophthalmologic imaging apparatus, image processing method, and storage medium

ABSTRACT

An image processing apparatus includes an identification unit configured to identify periodicity of a fundus image obtained by capturing an image of a fundus of an eye, and an information acquisition unit configured to acquire information indicating an imaging state of photoreceptor cells in the fundus image based on the periodicity.

BACKGROUND

Examination of a fundus of the eye is widely accepted as important inthe early diagnosis of lifestyle-related diseases and diseases likely tocause blindness. Fundus cameras and scanning laser ophthalmoscopes (SLO)are among the apparatuses used for the inspection of the fundus of theeye. The fundus camera captures an image of a fundus of the eye byreceiving reflected light of a light beam which has entered the fundusof the eye. The SLO is an ophthalmologic apparatus that uses theprinciple of confocal laser scanning microscope. In recent years, funduscameras and SLOs including an adaptive optical system have beendeveloped and are used for acquiring fundus planar images of highlateral resolution. The adaptive optical system measures an aberrationof a subject's eye by a wavefront sensor in real time and corrects theaberration of the measuring beam and the return beam that occurs at thesubject's eye by a wavefront correcting device. Further, attempts arebeing made to capture images of photoreceptor cells of a retina using bythese apparatuses and make a diagnosis of a disease or evaluate drugresponse.

As an example of visualization of the photoreceptor cells using anadaptive optics SLO, Kaccie Y. Li and Austin Roorda, “Automatedidentification of cone photoreceptors in adaptive optics retinal images”J. Opt. Soc. Am. A, May 2007, Vol. 24, No. 5, 1358 discusses anophthalmologic imaging apparatus which is capable of automatedextraction of photoreceptor cells by acquiring a planar image of thefundus of the eye regarding the retina. According to this technique, afundus planar image of a retina with high lateral resolution is acquiredby preprocessing the acquired planar image, in other words, removinghigh frequency components from the planar image using periodicity of thearrangement of the photoreceptor cells visualized in the image.

SUMMARY

According to some embodiments of the present invention, an imageprocessing apparatus includes an identification unit configured toidentify periodicity of a fundus image obtained by capturing an image ofa fundus of an eye, and an information acquisition unit configured toacquire information indicating an imaging state of photoreceptor cellsin the fundus image based on the periodicity.

Further features and aspects of the present disclosure will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the embodiments of the present invention.

FIG. 1 illustrates a functional configuration of an image processingapparatus according to a first exemplary embodiment.

FIG. 2 is a flowchart illustrating processing procedures of the imageprocessing apparatus according to the first exemplary embodiment.

FIG. 3 is a schematic view of a high-precision planar image of a fundusof the eye including photoreceptor cells acquired by an adaptive opticsSLO apparatus.

FIG. 4 illustrates an example of a Fourier image obtained by frequencyconversion of the planar image of the fundus of the eye.

FIGS. 5A and 5B illustrate a method for calculating a structure thatreflects an arrangement of the photoreceptor cells from a Fourier imageand a graph organizing the result of the calculation.

FIGS. 6A and 6B illustrate an example of a Fourier image obtained byfrequency conversion of the planar image of the fundus of the eye in acase where the signal is weak and a graph indicating the arrangement ofthe photoreceptor cells.

FIG. 7 illustrates a feature quantity acquired from a Fourier image.

FIGS. 8A, 8B, and 8C illustrate an example of a Fourier image of lowresolution and feature quantities acquired from the image.

FIGS. 9A, 9B, and 9C illustrate an example of a Fourier image of evenlower resolution and feature quantities acquired from the image.

FIGS. 10A and 10B illustrate an image quality index acquired from afeature amount extracted from a Fourier image.

FIG. 11 is a functional configuration of the image processing apparatusaccording to a second exemplary embodiment.

FIG. 12 is a flowchart illustrating processing procedures of the imageprocessing apparatus according to the second exemplary embodiment.

FIG. 13 illustrates an example of dividing the planar image of thefundus of the eye into a plurality of local planar images of the fundusof the eye.

FIGS. 14A and 14B illustrate examples of indices acquired from the localplanar images of the fundus of the eye.

FIG. 15 illustrates a hardware configuration of the image processingapparatus according to another exemplary embodiment.

FIG. 16 illustrates a configuration of an ophthalmologic imaging system.

FIG. 17 illustrates a configuration of an ophthalmologic imagingapparatus.

FIGS. 18A, 18B, and 18C illustrate an image acquisition method foroptical coherence tomography (OCT).

FIGS. 19A, 19B, 19C, and 19D illustrate an image acquisition method forSLO.

DESCRIPTION OF THE EMBODIMENTS

In the following detailed description, for purposes of explanation,numerous specific details are set forth in order to provide a thoroughunderstanding of the disclosed embodiments. It will be apparent,however, that one or more embodiments may be practiced without thesespecific details. In other instances, well-known structures and devicesare schematically shown in order to simplify the drawing.

When images of photoreceptor cells are captured, it is useful toevaluate the imaging state of the image of the photoreceptor cells sincesuch information helps the adjustment of the imaging conditions and thediagnosis of the obtained image. Generally, in evaluating an imagingstate, an image quality index value is obtained according to acomparison of a noise level of a region being an evaluation target and anoise level of a different region. However, since the imaging area ofthe image of the photoreceptor cells is generally the retina region, thecomparison method cannot be used for the evaluation.

In one embodiment of the present invention, an index that objectivelyevaluates an image quality of an image of photoreceptor cells of aretina acquired by an ophthalmologic apparatus such as an adaptiveoptics SLO is presented.

According to an exemplary embodiment of the present invention, sinceinformation of the imaging state of the photoreceptor cells can beobtained from processing of an image of the fundus of the eye, theadjustment of the imaging conditions and the diagnosis of the image canbe easily performed.

According to a first exemplary embodiment, when an image is obtained byimaging of photoreceptor cells of a retina by an adaptive optics SLO, anindex that quantitatively indicates the image quality of the acquiredimage is calculated and presented. More specifically, a spatialfrequency image is acquired by discrete Fourier transform using a planarimage of a fundus of the eye which has been acquired by the adaptiveoptics SLO. The acquired spatial frequency image is hereinafter referredto as a Fourier image or a frequency image. Then, a feature quantity ofa periodic structure that reflects the regular arrangement of thephotoreceptor cells is extracted from the acquired Fourier image, andthe index of the image quality is acquired from the extracted featurequantity.

If the image quality index acquired in such a manner is presented when auser is capturing an image of the fundus of the subject's eye, the usercan determine whether the image needs to be captured again. Further, ifa diagnosis is to be performed based on the density of the photoreceptorcells, the user can determine whether it is adequate to use such animage for the diagnosis.

FIG. 3 schematically illustrates a planar image of the fundus of the eyecaptured by the adaptive optics SLO. As illustrated in FIG. 3, a smallregion having relatively high luminance can be extracted in adistinguishable manner as a photoreceptor cell PR. Further, a bloodvessel region V having low luminance compared to the luminance of thephotoreceptor cell may be extracted. The blood vessel region V is theshadow of the blood vessel above the photoreceptor cells.

FIG. 4 illustrates an example of a Fourier image acquired by thediscrete Fourier transform of spatial frequency components of the planarimage of the fundus of the eye described above. As illustrated in FIG.4, a ring that corresponds to the periodicity of the photoreceptor cellsis formed according to the periodic arrangement of the photoreceptorcells.

FIG. 1 illustrates a functional configuration of an image processingapparatus 10 according to the present embodiment.

An image acquisition unit 100 in FIG. 1 acquires a planar image of afundus of the eye. The acquired planar image is an image ofphotoreceptor cells of the fundus of the eye. For example, it is animage acquired by a fundus imaging apparatus which includes anaberration measurement unit such as a Hartmann-Shack wavefront sensorand an adaptive optical system which corrects the aberration. Theadaptive optical system includes, for example, a reflective displaypanel having liquid crystals arranged on a silicon substrate (liquidcrystal on silicon (LCOS)) or a deformable mirror.

When the planar image of the fundus of the eye is captured by theadaptive optics SLO, an input information acquisition unit 110 acquiresinformation of the subject's eye which is being captured. The acquiredimage is stored in a storage unit 130 via a control unit 120. An imageprocessing unit 140 includes a frequency conversion unit 141, a featureextraction unit 142, and an index calculation unit 143.

The frequency conversion unit 141 performs frequency conversion of theplanar image of the fundus of the eye and obtains a frequency image. Forexample, the discrete Fourier transform is used for convertingfrequencies, as described above.

The feature extraction unit 142 extracts the feature quantity indicatingthe periodic structure of the planar image of the fundus of the eye fromthe frequency image. In the frequency image, the periodic structure ofthe planar image of the fundus of the eye is, for example, appears in aring structure having a point of origin at the center. The ring of thefrequency image presents a specified frequency band corresponding to thestructure of the photoreceptor cells.

The feature quantity of the periodic structure is, for example, thefeature quantity concerning this ring structure such as a peak luminancevalue in the ring structure and a sharpness value that indicates theextension of the ring structure. Further, there is the peak position ofthe luminance value in the ring structure. Additionally, if the imagequality is poor, a disk-like structure may appear in the frequency imagerather than the ring structure. In this case, the extension and theposition of the disk region, and the peak luminance value in the diskregion are extracted as the feature quantities. These are used foracquiring information of the imaging state and the distributioninformation of the photoreceptor cells in the planar image of the fundusof the eye.

The periodicity can be identified using information other than thefrequency image. For example, the peak value of the luminance near thecenter of the photoreceptor cells is detected and an average of theprofile of luminance distribution around each detection point isacquired. In this manner, the periodicity of the image of the fundus ofthe eye can be identified.

The peak value can be detected using maximum value detection processing.The profile in this context is in the radial direction having thedetection point at the center, and the mean value is taken in theangular direction of the polar coordinates having the detection point atthe center. Then, the feature extraction unit 142 evaluates the shape.If the shape is periodic, a peak corresponding to a detection pointadjacent to the detection point appears on the average profile.

Additionally, as a method for identifying the periodicity, co-occurrencematrix and fractal dimension known as the texture feature quantity canbe used.

The identification of the periodicity by the Fourier transform is oneexemplary embodiment, and the above-described different methods can beused in identifying the periodicity. In this regard, the frequencyconversion unit 141 functions as one exemplary embodiment of anidentification unit of the periodicity.

The index calculation unit 143 functions as an information acquisitionunit configured to acquire information of the imaging state of thephotoreceptor cells and distribution information of the photoreceptorcells from the image indicating the periodicity such as a Fourier image.By using the feature quantity extracted by the feature extraction unit142, the index calculation unit 143 can obtain accurate calculationresults of the imaging state and the distribution of the photoreceptorcells.

An output unit 150 outputs the information of the imaging state and thedistribution information of the photoreceptor cells acquired by theindex calculation unit 143. The information is output to a display unit160 as well as an external database and an output apparatus. Theinformation is, for example, printed out by the output apparatus.Information of the imaging state can be displayed as it is or togetherwith the planar image of the fundus of the eye on the output apparatus.

As the information of the imaging state which is displayed, there is theimage quality index value of the planar image of the fundus of the eyewhere the photoreceptor cells have been captured, and information askingthe user to perform operation to improve the imaging state. Further, afrequency image including the information of the imaging state can bedisplayed together with the image of the fundus of the eye. Furthermore,as illustrated in FIG. 5B described below, a graph showing a relationbetween a distance from the center position of the ring structure orfrom the point of origin of the frequency image and the luminance valueis displayed. Additionally, a control value corresponding to the imagingstate of the photoreceptor cells is output to the ophthalmologic imagingapparatus which has captured the planar image of the fundus of the eye.

The image processing unit 140 generates a Fourier image from theacquired planar image of the fundus of the eye, calculates an index ofthe image quality from the feature quantity extracted from the Fourierimage, and stores the index in the storage unit 13. The output unit 150outputs the calculated index to a monitor. Further, the output unit 150outputs the result of the processing stored in the storage unit 130 tothe database.

Although the image processing apparatus 10 acquires the planar image ofthe fundus of the eye directly from the adaptive optics SLO in thepresent embodiment, the image can be acquired via a network. In such acase, a plurality of planar images of the fundus of the eye captured bythe adaptive optics SLO are stored in a database connected to theadaptive optics SLO via the network, and the image processing apparatus10 acquires the images from the database via the network.

Next, processing procedures of the image processing apparatus 10according to the present embodiment will be described with reference tothe flowchart of FIG. 2.

In step S210, the image acquisition unit 100 acquires a plurality ofplanar images of the fundus of the eye to be analyzed from the adaptiveoptics SLO connected to the image processing apparatus 10 or a databasewhere the planar images of the fundus of the eye captured by theapparatus are stored. The acquired planar images of the fundus of theeye are stored in the storage unit 130 via the control unit 120.

Further, the image acquisition unit 100 acquires imaging parameterinformation of the planar images of the fundus of the eye when they arecaptured and stores the information in the storage unit 130 via thecontrol unit 120. The imaging parameter information is, for example,position information of the fixation lamp when the imaging is performed.The imaging parameter information such as the position information ofthe fixation lamp may be included in an information file which is addedto the planar images of the fundus of the eye, but may also exist as taginformation of the images.

In step S220, the input information acquisition unit 110 acquiresinformation of the subject's eye from the database or from the input ofthe operator via an input unit (not illustrated). The information of thesubject's eye is information of the patient (e.g., patient ID, name,age, and sex), whether the examination target is right/left eye, andshooting date and time. The acquired information is stored in thestorage unit 130 via the control unit 120.

In step S230, the frequency conversion unit 141 acquires a spatialfrequency image by the discrete Fourier transform using the planarimages of the fundus of the eye acquired by the adaptive optics SLO andstored in the storage unit 130. As illustrated in FIG. 3, a greater partof each planar image of the fundus of the eye is populated withregularly-arranged photoreceptor cells observed as small regions withhigh luminance. Thus, even if the image partially includes a bloodvessel or a region of lesion, a Fourier image obtained from such aplanar image of the fundus of the eye by spatial frequency conversionhas a ring structure as illustrated in FIG. 4.

In step S240, the feature extraction unit 142 extracts a featurequantity of the ring structure that shows the periodicity of thearrangement of the photoreceptor cells from the Fourier image obtainedin step S230. More specifically, as illustrated in FIG. 5A, if theFourier image is a square image having a number N of pixels in thevertical and horizontal directions (i.e., a square of N×N pixels), polarcoordinates (r, θ) having the center of the Fourier image (coordinates(N/2, N/2)) as the point of origin is considered. Then, a function I(r),which is obtained by calculating the value of each pixel of the Fourierimage in the 8 direction, is calculated where r=0, 1, 2 . . . N/2. Sincethe Fourier image is not a continuous image and each pixel has a value,when the function I(r) is calculated, if r of each pixel is, forexample, 4.5 or greater and smaller than 5.5, the value of I(5) is used.Subsequently, the function I(r) is smoothed, for example, by acquiring amean value of adjacent points. FIG. 5B illustrates a function I(r)acquired from the Fourier image in FIG. 5A.

The function I(r) in FIG. 5B includes a lot of information regarding thearrangement of the photoreceptor cells. For example, when thecrystalline lens of the subject's eye becomes cloudy due to a disease,the signals of the photoreceptor cells become extremely weak (see FIG.6A). In such a case, as illustrated in FIG. 6B, the value of thefunction I(r) itself becomes small compared to the function I(r) in FIG.5B. Further, if the photoreceptor cells are partially absent in theimaging region and a periodic structure is not produced, a similarfunction I(r) is obtained. Thus, as a feature quantity indicating theintensity of the periodic structure of the photoreceptor cells, asillustrated in FIG. 7, Imax (i.e., a maximum value of I(r)) and Isum(i.e., a sum of I(r)) can be used.

$I_{\max} = {\max\limits_{r}{I(r)}}$$I_{sum} = {\sum\limits_{r}{I(r)}}$

Further, rmax, which is a value of r of Imax, corresponds to theperiodicity of the arrangement of the photoreceptor cells.

$r_{\max} = {\arg {\max\limits_{r}{I(r)}}}$

If the distance between adjacent photoreceptor cells is small and thephotoreceptor cells are densely arranged, rmax becomes greater.Conversely, if the photoreceptor cells are distant from the macula luteaand the density of the photoreceptor cells is low, rmax becomes smaller.

Further, even if the signal strength is strong enough, the periodicstructure of the photoreceptor cells may not be clearly determined dueto low resolution (see FIG. 8A). In such a case, as illustrated in FIG.8B, the shape of the function I(r) is broadened toward the smallerfunction r. As a feature quantity that represents such poor resolution,a ratio of a sum of values around rmax (±n) and a sum of values around rsmaller than rmax is calculated. FIG. 8C illustrates the calculationregions.

${peak\_ ratio} = {\frac{I_{sum\_ peak}}{I_{sum\_ low}} = \frac{\sum\limits_{r = {r_{\max} - n}}^{r_{\max} + n}{I(r)}}{\sum\limits_{r = {r_{\max} - {3n}}}^{r_{\max} - n}{I(r)}}}$

If the imaging condition is poor and the resolution is low (FIG. 9A), itmay be difficult to acquire the peak I(r). FIG. 9B illustrates thesignals of I(r) in such a case. In such a case, even if Imax isacquired, accurate peak_ratio cannot be calculated. In such a case, apeak position which is assumed from the acquired visual fixationinformation (position r_expected_peak) will be used in place of rmax incalculating the peak_ratio. In this manner, the resolution of the imagecan be evaluated more accurately.

In step S250, from the feature quantity acquired in step S240, the indexcalculation unit 143 acquires information of the photoreceptor cellssuch as the information of the imaging state and the distributioninformation of the photoreceptor cells of the image captured by theadaptive optics SLO. The acquired image quality index is stored in thestorage unit 130 via the control unit 120. An example of a method forcalculating the image quality index from the feature quantity acquiredin step S240 will be described below. The index which is acquired,however, is not limited to the calculation method described below.

Among a plurality of indices extracted in step S240, the indicesassociated with the intensity of the periodic structure of thephotoreceptor cell such as Imax and Isum and those associated with theresolution such as the peak_ratio are important in terms of imagequality. Regarding the adaptive optics SLO, even if the health level ofthe subject's eye is good and high signal can be obtained, goodresolution is not always obtained.

In such a case, an image of good quality may be acquired if, forexample, the aberration is corrected again using the wavefront sensor.Thus, it is important to differentiate such an image from an image of asubject's eye with a cloudy crystalline lens less likely to produce goodimage. Thus, by combining the index associated with the intensity of theperiodic structure and the index associated with the resolution in atwo-dimensional manner, information which can more accurately presentthe imaging state can be acquired compared to a case where informationis acquired from each index.

FIGS. 10A and 10B illustrate examples of the index. In FIGS. 10A and10B, the vertical axis represents Imax, which is an index associatedwith the intensity of the periodic structure and the horizontal axisrepresents the peak_ratio, which is an index associated with theresolution. In step S240, a case where rmax is used for calculating thepeak_ratio and another case where the position r_expected_peak is usedfor calculating the peak_ratio have been described. Generally, when thesignal strength is reduced, the noise is increased, and it becomesdifficult to acquire an accurate rmax. Thus, in FIGS. 10A and 10B, acase where the peak_ratio is calculated using the r_expected_peak ispresented.

As illustrated in FIG. 10B, an image in a region A, which is a highindex region regarding the intensity of the periodic structure and theresolution, is considered as an image of good quality. An image in aregion B, which is a low index region regarding the intensity of theperiodic structure and the resolution, is considered as an image of poorquality. Regarding an image in a region C, which is a high index regionregarding the intensity of the periodic structure but a low index regionregarding the resolution, the state of the subject's eye is not bad butthe resolution may be poor. If such an image is acquired, by performingthe aberration correction again, the image quality may be improved. Suchan image may also include both a region where the structure of thephotoreceptor cells is clear a region where the structure of thephotoreceptor cells is not clear. Such a portion may correspond to alesion portion. Further, an image in a region D, which is a low indexregion regarding the intensity of the periodic structure and a highindex region regarding the resolution, has good resolution but weaksignals in general. If an image is such, a clearer image may be acquiredby increasing the signal strength.

In step S260, the output unit 150 acquires the information indicatingthe imaging state, such as the image quality index, stored in thestorage unit in step S250 and displays it on the display unit 160.

For example, the acquired feature quantity is displayed as it is as theinformation of the imaging state. Although detailed information can bepresented to the user, if a threshold value of the image quality indexis set in advance when the image index value is smaller than thethreshold value, a message such as “low image quality of photoreceptorcell region” can be displayed on the display unit 160 by the output unit150 as the information of the imaging state. The information is notnecessarily character information and an icon or an image correspondingto the character information may be displayed in place of the characterinformation.

In this manner, for example, even if the inspector is not used to theoperation, the inspector can easily notice that the image has a qualityproblem. If the image index value exceeds the threshold value, a messagesuch as “good image quality” may be displayed. Then, the user canunderstand that the image has no quality problems.

Further, if the value of the peak_ratio is low and the value of the Imaxis high as described above, the output unit 150 causes the display unit160 to display a message indicating that the ophthalmologic imagingapparatus needs correction of aberration. According to this message, theuser can understand that the correction of aberration is necessary.Further, if the value of the peak_ratio is high and the value of theImax is low, a message indicating that an increase in imaging lightquantity is necessary is displayed on the display unit 160. In thismanner, the user can understand that a light quantity adjustment isnecessary.

Further, as information other than the information of the imaging state,the density information of the photoreceptor cells may be displayedbased on the position of the rmax. Further, a character or a graphicthat notifies the user that a blood vessel or a lesion region isextracted instead of the photoreceptor cells on the planar image of thefundus of the eye may be displayed.

The output unit 150 also causes the display unit 160 to display afrequency image, such as the one illustrated in FIG. 4, together withthe image of the fundus of the eye. Further, a graph, such as the oneillustrated in FIG. 7, may also be displayed. In this manner, detailedinformation of the imaging state can be presented to the user.Additionally, information of a distribution state of the photoreceptorcells such as density information may be displayed on the display unit.This information is based on the diameter of the ring structureextracted by the feature extraction unit 142. Further, a messageinforming the user that the image includes a region not includingphotoreceptor cells (i.e., a blood vessel or a lesion portion) can bedisplayed on the display unit.

Feature quantity and other information stored in the storage unit 130 insteps S210 to S250 are stored in a database.

According to the above-described configuration, an index useful inobjectively evaluating the image quality of a planar image of a fundusof the eye captured by the adaptive optics SLO apparatus can bepresented. According to the presentation of such an index, for example,when a diagnosis or a determination of an effect of a treatment isperformed using the density of the photoreceptor cells, an objectivecriterion for determination can be presented.

According to the first exemplary embodiment, a Fourier image is obtainedby a frequency conversion of the entire planar image of the fundus ofthe eye obtained by the adaptive optics SLO, and an index useful forevaluating the image quality of the entire planar image of the fundus ofthe eye is calculated by extracting various feature quantities from theobtained Fourier image. However, since the image quality of the planarimage of the fundus of the eye is not always consistent and the lesionportions may be unevenly distributed in one planar image, the intensityof the periodicity structure of the photoreceptor cells may be differentdepending on the portion.

According to a second exemplary embodiment, in order to obtain such alocal difference, the planar image of the fundus of the eye is segmentedinto a plurality of local regions. Then, a Fourier image is acquired foreach of the planar images of the fundus of the eye acquired by thesegmentation, and the planar image is analyzed using the featurequantities extracted from the Fourier images.

FIG. 11 illustrates a functional configuration of the image processingapparatus 10 according to the present embodiment. Since the functionalconfigurations of units other than the image processing unit 140 aresimilar to those illustrated in FIG. 1, their descriptions are notrepeated. According to the present embodiment, the image processing unit140 further includes an image segmentation unit 1140 and an integrationunit 1144 as well as the frequency conversion unit 141, the featureextraction unit 142, and the index calculation unit 143. After theplanar image of the fundus of the eye is segmented into a plurality ofregions, a feature quantity of each region is extracted. The acquiredfeature quantities are integrated into an integrated index. Then, theimage is evaluated according to the integrated index.

The image segmentation unit 1140 segments the planar image of the fundusof the eye into a plurality of partial regions. The frequency conversionunit 141 performs the frequency conversion for each partial region andobtains a partial frequency image. The feature extraction unit 142extracts a feature quantity based on the ring structure that appears onthe partial frequency image. The index calculation unit 143 acquires theinformation of the photoreceptor cells in the plurality of partialregions.

By obtaining the frequency image for each partial region, if a regionwhere the photoreceptor cells are not extracted is included in theoriginal image, the image can be segmented into a partial region where,for example, a blood vessel is extracted but photoreceptor cells are notextracted, and a partial region other than such a region. Extremely fineblood vessels and lesion portions buried in the noise and unable todetermine from the feature quantity of the frequency image are notconsidered in the segmentation.

Processing procedures of the image processing apparatus 10 according tothe present embodiment is described with reference to the flowchart ofFIG. 12. Since the processing procedures in steps S210, S220, S230,S240, and S250 are similar to the processing procedures describedaccording to the first exemplary embodiment, their descriptions are notrepeated.

According to the first exemplary embodiment, the calculation of theimage quality index is performed with respect to the entire planar imageof the fundus of the eye acquired by the adaptive optics SLO. Accordingto the present embodiment, the planar image of the fundus of the eye issegmented into a plurality of local regions, and the index of eachregion is calculated. Then, the indices are combined and used for theevaluation of the entire image. Thus, the image processed in steps S230,S240, and S250 is a local planar image of the fundus of the eye obtainedby the segmentation of the planar image of the fundus of the eye.

Next, each step will be described in detail.

In step S1230, the image segmentation unit 1140 segments the planarimage of the fundus of the eye acquired by the adaptive optics SLO andstored in the storage unit 130 into a plurality of local regions.Various methods can be used for the segmentation. Although the localdifference can be more noticeable if the planar image of the fundus ofthe eye is segmented into a great number of small regions, the accuracyof the information obtained from each local region will be reduced.Further, since the frequency conversion of a plurality of regionsrequires more time and cost, it is important to use an image size ofdata of 2 to the n-th power suitable for high speed Fourier transform.In the present embodiment, for example, with respect to the originalplanar image of the fundus of the eye having a size of 400×400, aplurality of local planar images of the fundus of the eye with a size of128×128 are acquired as described in FIG. 13. Each of the local planarimages includes an overlapping portion. The segmentation method,however, is not limited to such an example.

Sixteen local planar images of the fundus of the eye generated in thismanner are stored in the storage unit 130 via the control unit 120.Although the processing in steps S230, S240, and S250 is similar to theprocessing in the first exemplary embodiment, it is different in thatthe processing is performed for each of sixteen local planar images ofthe fundus of the eye generated in step 1230. Accordingly, sixteenindices are obtained. Further, each of the obtained indices and thefeature quantities acquired in the course of process is stored in thestorage unit 130 in association with the corresponding local planarimage of the fundus of the eye.

In step S1260, the integration unit 1144 generates an index byintegrating the indices acquired from each local planar image of thefundus of the eye. Then, the obtained index is displayed on the monitorvia the output unit 150. Further, the feature quantity and otherinformation stored in the storage unit 130 in steps S210 to S1260 arestored in the database.

FIGS. 14A and 14B illustrate examples of sixteen indices acquired instep S250. In FIG. 14A, all of the sixteen indices show poor imagequality. If the state of the subject's eye is not good and, for example,the crystalline lens is very cloudy, very few signals can be obtained.In such a case, all the indices acquired from the local planar images ofthe fundus of the eye show bad values. Thus, if indices such as thoseillustrated in FIG. 14A are obtained, the quality of the planar image ofthe fundus of the eye is considered as low as a whole.

On the other hand, if the indices of both high values and of low valuesare determined as illustrated in FIG. 14B, it can be considered thatsome of the local regions failed in the acquisition of the periodicstructure of the photoreceptor cells although the imaging itself hasbeen successful. Such a case indicates that the photoreceptor cells arenot extracted due to, for example, the existence of a blood vessel, orthe photoreceptor cells do not exist due to, for example, a lesion.

Regarding the extraction of a blood vessel from a planar image of afundus of the eye acquired by an adaptive optics SLO, there is a knownmethod such as the one discussed in Tam, J., Martin, J. A., Roorda, A.,“Non-invasive visualization and analysis of parafoveal capillaries inhumans” Invest. Ophthalmol. Vis. Sci. 51 (3): 1691-1698 (2010). Ifindices such as those illustrated in FIG. 14B are obtained according toa combination of such methods, whether the possibility of the localregion, indicating the low index value, having a blood vessel or alesion portion is high can be determined.

Further, the planar image can be evaluated by a combination of the firstand the second exemplary embodiments, for example, a combination of theoverall evaluation and the partial evaluation. By presenting a resultobtained from the combined evaluation, even if the overall evaluation isnot good, the evaluation will be useful since the user can determine thereason for the poor evaluation (e.g., poor imaging condition, lesionportion was included). This is useful as if any part of the partialevaluation indicates high image quality, the possibility of the imagingcondition being the cause of the poor image quality is low. Thus, anoptimum index can be presented by combining the evaluation methods.

According to the present embodiment, the planar image of the fundus ofthe eye acquired by the adaptive optics SLO apparatus is segmented intoa plurality of local regions. Further, the image quality indicesobtained from the local planar images of the fundus of the eye areintegrated to form an integrated index. According to this index, a casewhere the overall image quality of the fundus of the eye is low and acase where the image quality itself is not low but a region where theperiodic structure of the photoreceptor cells is not extracted existsdue to the presence of a blood vessel or a lesion region can bepresented.

Further, the ophthalmologic imaging apparatus can be configured such,from the aspect of hardware, that the index calculation unit 143automatically determines a lesion region. Then, based on the result ofthe determination, the output unit 150 outputs information regarding theimaging region to the apparatus. On receiving the information, theapparatus automatically captures the image again by setting the centerof the image to that imaging region.

If the output unit 150 outputs an aberration correction instruction or acontrol value of an adjustment value of a light quantity according tothe imaging state to the ophthalmologic imaging apparatus, then theophthalmologic imaging apparatus controls the aberration measurementunit and the correction optical system based on the input control value.Accordingly, the user does not need to adjust the apparatus.

According to the first exemplary embodiment, the information of theimaging state is displayed and the user performs the operation accordingto the displayed information. According to a third exemplary embodiment,the information of the imaging state is used for forming and selectingan image. Since the configurations of the apparatus are similar to thosedescribed in the first and the second exemplary embodiments, theirdescriptions are not repeated and the points different from theabove-described exemplary embodiments will be mainly described.

The image acquisition unit 100 acquires a plurality of planar images ofthe fundus of the eye obtained by imaging of different or substantiallysame positions. Since the fundus of the eye moves due to the involuntaryeye movement such as a saccade, even if a tracking function is provided,the tracking may not be successful. Thus, even if the apparatus is setto capture an image of the same position, actually, an image of adifferent position may be captured. This is why the expression“substantially the same” is used.

The image processing unit 140 functions as a determination unitconfigured to determine the image quality. The image processing unit 140acquires the information of the imaging state acquired by the indexcalculation unit 143 and the information of the threshold value of theimaging state determined in advance. Then, the image processing unit 140determines whether the imaging state obtained for each image exceeds offalls below the threshold value. As the information of the imagingstate, for example, the values of the peak_ratio and Imax described inthe first exemplary embodiment or an image quality index value obtainedby a combination of such values can be used.

Additionally, the image processing unit 140 functions as a selectionunit configured to select an image. If an image is determined by thedetermination unit as having an image quality below the threshold value,the image processing unit 140 determines that it is inappropriate forthe presentation to the user or not suitable for the superimposingprocessing and does not select the image. In other words, the imageprocessing unit 140 selects only the images that exceed the thresholdvalue. Since the images below the threshold value are not selected,images with aberration, images with insufficient light quantity, andimages captured when the eye instantaneously moved due to blinking orthe like are not selected by the image processing unit 140.

The image processing unit 140 further aligns the selected images andperforms the superimposing processing. Thus, the image processing unit140 also functions as a generation unit configured to generate an imagewith reduced random noise. Accordingly, the image quality of a pluralityof images can be automatically determined and a planar image of a fundusof the eye with good image quality can be obtained.

The control unit 120 outputs only the image selected from the acquiredplurality of images to the output unit 150. The control unit 120 outputssuch an image together with the generation unit or in place of thegeneration unit. The output unit 150 causes the display unit 160 todisplay only the selected image. The unselected images are nottransmitted to the output unit 150 but is stored in the storage unit130.

The unselected images can be deleted as a failed image with the approvalof the user. In addition to the display of the image for the diagnosis,the output unit 150 can separately display the images which have notbeen selected on the display unit 160 and further display a window thataccepts the approval of the user regarding the deletion. The controlunit 120 can delete the approved images from the storage unit 130 in oneoperation. Further, in place of the deletion, the control unit 120 cancause only the approved images not to be transferred to the externaldatabase. In this manner, the user can automatically determineunnecessary images and can easily perform the deletion processing or thenon-transfer processing.

The function of the above-described image processing apparatus 10 can berealized by the hardware illustrated in FIG. 15 and using the softwareand hardware together.

The image processing apparatus illustrated in FIG. 15 includes a centralprocessing unit (CPU) 1501, a random access memory (RAM) 1502, a readonly memory (ROM) 1503, an external storage unit 1504, and acommunication interface 1505. These units are connected to each othervia a bus 1509. Further, a monitor 1506, a keyboard 1507, and a mouse1508 are connected to the image processing apparatus 10. A programincluding an instruction that can cause the processing illustrated inthe flowchart of FIG. 2 or 12 is stored in the ROM 1503 or the externalstorage unit 1504.

The function of the image processing apparatus 10 of the above-describedexemplary embodiment is realized by the CPU 1501 reading out a storedprogram, loading it into the RAM 1502, and executing the instructionincluded in the program.

An example of an imaging system which acquires a planar image of thefundus of the eye described above will be described with reference toFIG. 16. The image processing apparatus 10 in the imaging system isconnected to an ophthalmologic imaging apparatus 30 and a database 50via a local area network (LAN) 40. The image processing apparatus 10 canbe wirelessly connected to the LAN 40. Further, the display unit 160 isconnected to the image processing apparatus 10.

The configuration of the imaging system is not limited to theabove-described example. For example, an ophthalmologic imagingapparatus which includes an ophthalmologic imaging unit, an imageprocessing unit, and the display unit 160 may be used. In this case, theophthalmologic imaging unit includes the function of the ophthalmologicimaging apparatus 30 and the image processing unit includes the functionof the image processing apparatus 10. If the imaging system has such aconfiguration, the quality of the image of the photoreceptor cells canbe determined by the ophthalmologic imaging apparatus 30 alone, and thesystem can be downsized.

Configuration of the ophthalmologic imaging apparatus 30 will bedescribed with reference to FIG. 17.

A composite apparatus composed of an SLO apparatus and an OCT apparatusaccording to the embodiments of the present invention is described as anophthalmologic imaging apparatus. In particular, the ophthalmologicimaging apparatus 30 which includes an adaptive optical system and cancapture both planar images (SLO images) of high lateral resolution aswell as tomographic images (OCT images) for a retina and acquires aplanar image of a fundus of the eye will be described. Theophthalmologic imaging apparatus 30 includes an SLO apparatus and an OCTapparatus. The SLO apparatus acquires a planar image of a fundus of theeye by correcting the optical aberration of the subject's eye and usinga spatial light modulator. The OCT apparatus employs Fourier domainimaging when it acquires a tomographic image. According to theseapparatuses, the ophthalmologic imaging apparatus 30 can obtain a goodplanar image of the fundus of the eye and a tomographic image of thesubject's eye regardless of the visibility and the optical aberration.

First, an overall configuration of the ophthalmologic imaging apparatus30 according to the present embodiment will be described in detail withreference to FIG. 17. Light emitted from a light source 201 is dividedinto a reference beam 205 and a measuring beam 206 by an optical coupler231. The measuring beam 206 is guided to a subject's eye 207 as anobject to be observed via a single mode fiber 230-4, a spatial lightmodulator 259, an XY scanner 219, an X scanner 221, and sphericalmirrors 260-1 to 260-9.

The measuring beam 206 is reflected or scattered by the subject's eye207 being an object to be observed and returned as a return beam 208.The return beam 208 enters a detector 238 or a line sensor 239. Thedetector 238 converts the light intensity of the return beams 208 into avoltage signal. Then, based on the voltage signal, a planar image of thefundus of the eye of the subject's eye 207 is generated. Further, thereturn beam 208 is combined with the reference beam 205, and thecombined light is caused to enter the line sensor 239. Accordingly, atomographic image of the subject's eye 207 is formed. Furthermore, byusing a plurality of acquired tomographic images, a three-dimensionalcourse of blood vessels can be extracted.

Although a spatial light modulator is used as a device for correctingthe wavefront aberration, in the present embodiment, any device can beused so long as the wavefront aberration can be corrected. Thus, forexample, a variable shape mirror can be used.

Next, the periphery of the light source 201 will be described. The lightsource 201 uses a super luminescent diode (SLD) being a typicallow-coherent light source. The central wavelength and the bandwidth are830 nm and 50 nm, respectively. In order to acquire a planar image ofthe fundus of the eye with small speckle noise, a low-coherent lightsource is selected. Although the SLD is selected as a type of the lightsource, a different light source can be used so long as low coherentlight can be emitted. For example, an amplified spontaneous emission(ASE) light source can be used.

Further, near-infrared light is suitable as a wavelength from theviewpoint of measurement of eyes. Further, since the wavelength affectsthe lateral resolution of the acquired planar image of the fundus of theeye, a shorter wavelength is desirable. In the descriptions below, awavelength of 830 nm is used. Other wavelengths may be selecteddepending on the measuring portion of the object to be observed.

Light emitted from the light source 201 is divided into the referencebeam 205 and the measuring beam 206 at a ratio of 96:4 via a single modefiber 230-1 and the optical coupler 231. The composite apparatus 30further includes polarization controllers 253-1 and 253-2.

Next, an optical path of the reference beam 205 will be described.

The reference beam 205 divided by the optical coupler 231 is adjusted sothat it is guided to a lens 235-1 via a single mode fiber 230-2 andadjusted so as to become parallel light with a beam diameter of 4 mm.

Next, the reference beam 205 is guided to a mirror 214 being a referencemirror by mirrors 257-1 to 257-4. Since the optical path length of thereference beam 205 is adjusted to be substantially the same as that ofthe measuring beam 206, the reference beam 205 and the measuring beam206 interfere with each other. Next, the reference beam 205 is reflectedby the mirror 214 and guided again to the optical coupler 231. Adispersion compensation glass 215, through which the reference beam 205passes, compensates the dispersion that occurs when the measuring beam206 travels to the subject's eye 207 and returns from the subject's eye207 with respect to the reference beam 205. In the followingdescriptions, for example, an average diameter of an oculus of aJapanese being L1=23 mm is used.

A motorized stage 217-1 is movable in the directions indicated by anarrow to allow the optical path length of the reference beam 205 to beadjusted and controlled. Further, the motorized stage 217-1 is driven bya motorized stage driver 283 in a driver unit 281 under the control of apersonal computer 225.

Next, the optical path of the measuring beam 206 will be described. Themeasuring beam 206 split by the optical coupler 231 is guided to a lens235-4 via the single mode fiber 230-4 and adjusted so as to becomeparallel light with a beam diameter of 4 mm. Further, the polarizationcontroller 253-1 or 253-2 can adjust the polarization state of themeasuring beam 206. In the present embodiment, the polarizationcontroller 253-1 or 253-2 adjusts the polarization state of themeasuring beam 206 to be linearly polarized in a direction parallel tothe drawing surface of FIG. 17.

The measuring beam 206 passes through a beam splitter 258 and a movablebeam splitter 261 (also referred to as a splitting unit) and reaches thespatial light modulator 259 via the spherical mirrors 260-1 and 260-2.Then, the measuring beam 206 is modulated at the spatial light modulator259. In the present embodiment, the spatial light modulator 259 utilizesthe orientation characteristics of the liquid crystal. Morespecifically, the spatial optical modulator 259 is arranged in adirection where the spatial optical modulator 259 can modulate the phaseof linear polarization parallel to the drawing surface of FIG. 17 (i.e.,the P polarization) to coincide with the polarization orientation of themeasuring beam 206.

Further, the measuring beam 206 passes through a polarizing plate 273and reaches a mirror of the X scanner 221 via the spherical mirrors260-3 and 260-4. In the present embodiment, the polarizing plate 273 hasa role to guide only the linear polarization parallel to the drawingsurface of FIG. 17, of the return beams 208, to the spatial lightmodulator 259. Further, in the present embodiment, the X scanner 221scans the measuring beam 206 in a direction parallel to the drawingsurface of FIG. 17. For example, the X scanner 221 is a resonant scannerhaving a drive frequency of approximately 7.9 kHz.

Further, the measuring beam 206 is incident on a mirror of the XYscanner 219 via the spherical mirrors 260-5 and 260-6. Although thenumber of the mirrors in the XY scanner 219 is illustrated as one,actually, the XY scanner 219 includes two mirrors (X-scanning mirror andY-scanning mirror) arranged close to each other. Further, the measuringbeam 206 is adjusted in such a manner that its center coincides with thecenter of rotation of the mirror of the XY scanner 219. The drivefrequency of the XY scanner 219 is changeable in the range up to 500 Hz.

The spherical mirrors 260-7 to 260-9 are optical systems that cause themeasuring beam 203 to scan a retina 227. Having a point near a cornea226 as a support point, the measuring beam 206 scans the retina 227.Although the measuring beam 206 has a beam diameter of 4 mm, the beamdiameter may be increased for acquisition of a high resolutiontomographic image.

A motorized stage 217-2 is configured to move in a direction indicatedby an arrow in FIG. 17 and to adjust and control the position of aspherical mirror 260-8 attached thereto. Similarly to the motorizedstage 217-1, the motorized stage 217-2 is controlled by the motorizedstage driver 283.

By adjusting the position of the spherical mirror 260-8, it is possibleto focus the measuring beam 206 on a predetermined layer of the retina227 of the subject's eye 207 and observe the subject's eye 207. In theinitial state, the position of the spherical mirror 260-8 is adjusted sothat the measuring beam 206 can enter the cornea 226 as parallel light.The ophthalmologic imaging apparatus 30 according to the presentembodiment can cope with the subject's eye 207 having a refractiveerror.

When the measuring beam 206 enters the subject's eye 207, the measuringbeam 206 becomes the return beam 208 due to reflection or scatteringfrom the retina 227. Then, the return beam 208 reaches the line sensor239 by being guided again by the optical coupler 231. A part of thereturn beam 208 is reflected by the movable beam splitter 261 and guidedto the detector 238 via a lens 235-5.

A light blocking plate 272 having a pinhole has a function of blockingunnecessary light not focused on the retina 227 among the return beams208. Further, the light blocking plate 272 is arranged in a positionconjugate with the focusing position of the lens 235-5. The diameter ofthe pinhole of the light blocking plate 272 is, for example, 50 μm. Asthe detector 238, for example, an avalanche photodiode (APD), which is ahigh-speed, highly-sensitive light sensor, is used. Apart of the returnbeam 208 split by the beam splitter 258 enters a wavefront sensor 255.The wavefront sensor 255 is a Shack-Hartmann wavefront sensor.

The spherical mirrors 260-1 to 260-9 are arranged so that they areoptically conjugate with the XY scanner 219, the X scanner 221, thecornea 226, the wavefront sensor 255, and the spatial light modulator259. Thus, the wavefront sensor 255 can measure the aberration of thesubject's eye 207. Further, the spatial light modulator 259 can correctthe aberration of the subject's eye 207. Furthermore, by controlling thespatial light modulator 259 in real time based on the obtainedaberration, it is possible to correct the aberration that occurs in thesubject's eye 207 and to acquire tomographic images with higher lateralresolution.

Next, the configuration of the measurement system will be described. Theophthalmologic imaging apparatus 30 can acquire tomographic images (OCTimages) as well as planar images of the fundus of the eye (SLO images).

First, the measurement system for the tomographic images will bedescribed. The return beams 208 are combined by the optical coupler 231.The combined light (combined light 242) is guided to a transmissivegrating 241 via a single mode fiber 230-3 and a lens 235-2, and isdispersed for each wavelength. Then, the light enters the line sensor239 via a lens 235-3.

The line sensor 239 converts the light intensity for each position(wavelength) into a voltage signal. The voltage signal is converted intoa digital value by a frame grabber 240 so that tomographic images of thesubject's eye 207 are formed in the personal computer 225. The linesensor 239 includes 1024 pixels and can obtain the intensity of thecombined light 242 for each wavelength (a segmentation of 1024).

Next, the measurement system for the planar image of the fundus of theeye will be described. A part of the return beam 208 is reflected by themovable beam splitter 261. After unnecessary light is blocked by thelight blocking plate 272, the reflected light reaches the detector 238,and the light intensity is converted into an electric signal. Data ofthe obtained electric signal is processed by the personal computer 225in synchronization with the scanning signal of the X scanner 221 and theXY scanner 219. Accordingly, a planar image of the fundus of the eye isformed.

A part of the return beam 208 split by the beam splitter 258 enters thewavefront sensor 255, and the aberration of the return beam 208 ismeasured. An image signal obtained by the wavefront sensor 255 issupplied to the personal computer 225, and the aberration is calculated.The obtained aberration is expressed by Zernike polynomial, whichrepresents the aberration of the subject's eye 207. The Zernikepolynomial includes a tilt term, a defocus term, an astigmatism term, acoma term, and a trefoil term.

Next, an acquisition method for tomographic images (OCT images) usingthe ophthalmologic imaging apparatus 30 will be described with referenceto FIGS. 18A to 18C. The ophthalmologic imaging apparatus 30 acquires atomographic image of the retina 227 by controlling the XY scanner 219and acquiring an interference fringe by the line sensor 239 using the Xscanner 221 as a fixed mirror. The movable beam splitter 261 iscontrolled so that the return beams 208 are not guided to the detector238. Further, the X scanner 221 and the XY scanner 219 are controlled byan optical scanner driver 282 in the driver unit 281 from the personalcomputer 225. In the present embodiment, the method for acquiring atomographic image (a plane parallel to the optical axis) of the retina227 will be described.

FIG. 18A is a schematic view of the subject's eye 207 and illustrates astate where the subject's eye 207 is observed by the ophthalmologicimaging apparatus 30. As illustrated in FIG. 10A, when the measuringbeam 206 enters the retina 227 via the cornea 226, the measuring beam206 becomes the return beams 208 due to reflection or scattering atvarious positions. Then, the return beam 208 reaches the line sensor 239with delays at the respective positions.

Since the light source 201 has a large bandwidth and a short coherencelength, the line sensor 239 can detect an interference fringe when theoptical path length of the reference beam path is approximately equal tothe optical path length of the measuring beam path. As described above,the interference fringe acquired by the line sensor 239 is aninterference fringe in a spectrum region on the wavelength axis.

Subsequently, the interference fringe, which is information of thewavelength axis, is converted into an interference fringe on an opticalfrequency axis considering the characteristics of the line sensor 239and the transmissive grating 241. Further, by performing inverse Fouriertransform of the obtained interference fringe on the optical frequencyaxis, it is possible to obtain information in the depth direction. Asillustrated in FIG. 18B, if the interference fringe is detected whilethe XY scanner 219 is driven, the interference fringe can be obtained ateach position on the X-axis. More specifically, the information in thedepth direction can be obtained at each position on the X-axis. As aresult, a two-dimensional distribution of the intensity of the returnbeams 208 on the XZ plane is obtained, which is a tomographic image 232(see FIG. 18C).

As described above, the tomographic image 232 is an image obtained byarranging the intensity of each of the return beams 208 in an array. Forexample, the tomographic image 232 is a gray scale intensity image. Thelength of the tomographic image 232 in the X direction is 700 μm, whichis similar to the SLO image described below.

In FIG. 18C, only the boundaries in the obtained tomographic image areexpressed in lines. The illustration includes a retinal pigmentepithelial layer 246, a stratum opticum 247, and a blood vessel 278.Further, if a plurality of tomographic images is acquired at eachposition on the Y-axis, it is possible to visualize a three-dimensionalcourse of the blood vessel.

Next, an acquisition method of a planar image of the fundus of the eye(SLO image) using the ophthalmologic imaging apparatus 30 will bedescribed. The ophthalmologic imaging apparatus 30 can acquire a planarimage of the fundus of the eye of the retina 227 by controlling andoperating the XY scanner 219 in only the Y-axis direction and the Xscanner 221 while fixing the X-axis direction of the XY scanner 219, andacquiring the intensity of the return beam 208 using the detector 238.

The X scanner 221 and the XY scanner 219 are controlled by the opticalscanner driver 282 in the driver unit 281 from the personal computer 225(see FIG. 17). Further, the ophthalmologic imaging apparatus 30 canacquire a planar image of the fundus of the eye while correcting theaberration that occurs at the subject's eye 207 by controlling thespatial light modulator 259. The spatial light modulator 259 can becontrolled by using the aberration of the subject's eye 207 measured bythe wavefront sensor 255. Furthermore, the ophthalmologic imagingapparatus 30 can acquire a planar image of the fundus of the eye whilecontrolling the spatial light modulator 259 in real time.

Next, the acquisition method of the planar image of the fundus of theeye (SLO image) will be described with reference to FIGS. 19A to 19D.

The ophthalmologic imaging apparatus 30 acquires a planar image of afundus of the eye of the retina 227 by controlling the XY scanner 219and acquiring the intensity of the return beam 208 by the detector 238.An acquisition method of a planar image of the fundus of the eye of theretina 227 (planar image perpendicular to the optical axis) will now bedescribed.

FIG. 19A is a schematic view of the subject's eye 207 and illustrates astate where the subject's eye 207 is observed by the ophthalmologicimaging apparatus 30.

As illustrated in FIG. 19A, when the measuring beam 206 enters theretina 227 via the cornea 226, the measuring beam 206 becomes the returnbeam 208 due to reflection or scattering at various positions. Then, thereturn beam 208 reaches the detector 238. Further, as illustrated inFIG. 19B, if the intensity of the return beam 208 is detected while theXY scanner 209 is moved in the X direction, information at each positionon the X-axis can be obtained.

As illustrated in FIG. 19C, the XY scanner 209 is simultaneously movedin both the X-axis and Y-axis directions with respect to an imagecapturing range 292 where the retina 227 is present. Then, the measuringbeam 206 is raster-scanned along a trajectory 293 on the image capturingrange 292. In this state, if the intensity of the return beams 208 isdetected, a two-dimensional distribution of the intensity of the returnbeams 208 can be obtained. Accordingly, a planar image of the fundus ofthe eye 277 (see FIG. 19D) is acquired.

In FIG. 19D, the measuring beam 206 is scanned from a point S at theupper right to a point E at the bottom left. While the scanning isperformed, the intensity of the return beams 208 is used in forming theplanar image of the fundus of the eye 277. The trajectory 293 from thepoint E to the point S is the movement of the measuring beam 206 as apreparation for the imaging of the planar image of the fundus of the eye277 which is to be formed next. The time required for the scanning takes84% for the point S to the point E and 16% for the point E to the pointS with respect to the trajectory 293 in FIG. 19C. This ratio is based onthe duty ratio of the drive waveform of the Y scanner described above.Further, in FIG. 19C, for the sake of simplicity, the number of times ofthe scanning in the X direction with respect to the trajectory 293 issmaller than the actual number of times.

The planar image of the fundus of the eye 277 has a size of 700×350 μm.The time required for the acquisition is approximately 15.6 ms. Thistime is based on the drive frequency of the Y scanner.

In the planar image 277 of the fundus of the eye, a photoreceptor cellgroup 279, where the intensity of the return beam 208 is relativelyhigh, is light colored, whereas the blood vessel 278, where theintensity is relatively low, is dark colored. Further, blood cells (notillustrated) are light colored in the blood vessel 278. If the planarimage of the fundus of the eye 277 is continuously acquired, themovement of the blood cells through the blood vessel 278 can bevisualized. Further, spatio-temporal images may be generated byextracting the blood vessel 278, from which the blood cells arevisualized, from the planar images 277 of the fundus of the eye whichare continuously acquired, and superimposing the extracted planar images277 of the fundus of the eye in the order they have been captured.Movement of the blood cells and the blood speed can be easily obtained.

An imaging method for obtaining an image useful for obtaining the bloodspeed and an imaging method for obtaining an image of the photoreceptorcells are designated by the user via an operation unit of the imageprocessing apparatus 10. When the user selects the imaging method viathe operation unit, a designation unit (not illustrated) of the imageprocessing apparatus designates the selected imaging method andtransmits associated information to the ophthalmologic imaging apparatus30. Then, according to the designated imaging method, the ophthalmologicimaging apparatus 30 outputs an image of photoreceptor cells which iseasy to observe or an image having a bloodstream as an object to beobserved.

The embodiment can also be realized by a computer of a system orapparatus (or devices such as a CPU or MPU) that reads out and executesa program recorded on a memory device to perform the functions of theabove-described embodiment(s), and by a method, the steps of which areperformed by a computer of a system or apparatus by, for example,reading out and executing a program recorded on a memory device toperform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable storage medium).

While the embodiment of the present invention has been described withreference to exemplary embodiments, it is to be understood that theinvention is not limited to the disclosed exemplary embodiments. Thescope of the following claims is to be accorded the broadestinterpretation so as to encompass all modifications, equivalentstructures, and functions.

This application claims priority from Japanese Patent Application No.2011-204653 filed Sep. 20, 2011, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: anidentification unit configured to identify periodicity of a fundus imageobtained by capturing an image of a fundus of an eye; and an informationacquisition unit configured to acquire information indicating an imagingstate of photoreceptor cells in the fundus image based on theperiodicity.
 2. The image processing apparatus according to claim 1,wherein the identification unit is configured to obtain a frequencyimage as information indicating the periodicity of the fundus image byperforming frequency conversion of the fundus image obtained bycapturing an image of the fundus of the eye.
 3. The image processingapparatus according to claim 1, further comprising a segmentation unitconfigured to segment the fundus image into a plurality of partialregions, wherein the identification unit is configured to obtain apartial frequency image by performing frequency conversion for each ofthe plurality of partial regions, and wherein the informationacquisition unit is configured to acquire information about thephotoreceptor cells in each of the plurality of partial regions.
 4. Theimage processing apparatus according to claim 3, wherein the informationacquisition unit is configured to acquire, based on a feature quantitycorresponding to each partial region, information about a partial regionincluding a region from which the photoreceptor cells are not extracted.5. The image processing apparatus according to claim 1, furthercomprising an extraction unit configured to extract a feature quantityindicating a periodic structure of the fundus image, wherein theinformation acquisition unit is configured to acquire informationindicating an imaging state of a region of the photoreceptor cells inthe fundus image based on the feature quantity.
 6. The image processingapparatus according to claim 1, further comprising an extraction unitconfigured to extract a feature quantity indicating a ring structure ofthe fundus image, wherein the information acquisition unit is configuredto acquire information indicating an imaging state of the photoreceptorcells in the fundus image based on the feature quantity.
 7. The imageprocessing apparatus according to claim 6, wherein the extraction unitis configured to extract a magnitude in luminance value of a peak of aring structure of a frequency image as information indicating theperiodicity as a feature quantity indicating the imaging state of thephotoreceptor cells in the fundus image.
 8. The image processingapparatus according to claim 6, wherein the extraction unit isconfigured to extract a sharpness of a ring structure of a frequencyimage as information indicating the periodicity as a feature quantityindicating the imaging state of the photoreceptor cells in the fundusimage.
 9. The image processing apparatus according to claim 6, whereinthe extraction unit is configured to identify a frequency band greaterthan a predetermined threshold value of the fundus image as the ringstructure.
 10. The image processing apparatus according to claim 5,further comprising: a determination unit configured to determine whetherthe imaging state is better or worse than a predetermined thresholdvalue; and a selection unit configured to select at least one fundusimage from a plurality of fundus images whose imaging state has beendetermined by the determination unit.
 11. The image processing apparatusaccording to claim 2, wherein the fundus image is an image of the fundusof the eye captured by an imaging method for obtaining an image of thephotoreceptor cells, and wherein the information acquisition unit isconfigured to acquire information indicating presence/absence of aregion where the photoreceptor cells are not extracted and a bloodvessel or a lesion is extracted from the fundus image based on thefrequency image.
 12. The image processing apparatus according to claim1, further comprising an output unit configured to output theinformation indicating the imaging state of the photoreceptor cells. 13.The image processing apparatus according to claim 12, wherein the outputunit is configured to cause a display unit to display the fundus imageand the information indicating the imaging state of the photoreceptorcells.
 14. The image processing apparatus according to claim 13, whereinthe information acquisition unit is configured to acquire theinformation indicating the imaging state of the photoreceptor cells inthe fundus image, and wherein the output unit is configured to cause,based on the information indicating the imaging state, the display unitto display the fundus image and at least one of information indicatingthat an aberration correction is required at an imaging unit for thefundus image and information indicating that an increase in quantity ofimaging light emitted from the imaging unit for the fundus image isrequired.
 15. The image processing apparatus according to claim 13,wherein the output unit is configured to cause the display unit todisplay the fundus image and a frequency image as information indicatingthe periodicity.
 16. The image processing apparatus according to claim12, wherein the output unit is configured to cause a display unit todisplay a graph indicating a relation between a luminance value and adistance from a specified position in a frequency image as informationindicating the periodicity.
 17. The image processing apparatus accordingto claim 12, wherein the output unit is configured to output a controlvalue corresponding to the imaging state of the photoreceptor cells toan imaging apparatus configured to capture an image of the fundus of theeye.
 18. The image processing apparatus according to claim 1, whereinthe fundus image is an image of the fundus of the eye obtained byfocusing on a predetermined depth position of the fundus of the eye by afundus imaging apparatus configured to correct an aberration by anaberration measurement unit and an adaptive optical system.
 19. An imageprocessing apparatus comprising: a conversion unit configured to acquirea frequency image by performing frequency conversion of a fundus imageobtained by capturing an image of a fundus of an eye; an extraction unitconfigured to extract a feature quantity of a ring structure thatappears in the frequency image; and an information acquisition unitconfigured to acquire information about photoreceptor cells in thefundus image based on the feature quantity.
 20. An ophthalmologicimaging apparatus comprising: a designation unit configured to designatean imaging method for acquiring an image of photoreceptor cells of asubject's eye; an imaging unit configured to acquire a fundus image bycapturing an image of a fundus of the subject's eye according toinformation on the imaging method; an identification unit configured toidentify periodicity of the fundus image; and an information acquisitionunit configured to acquire information indicating a distribution of thephotoreceptor cells in the fundus image based on the periodicity. 21.The ophthalmologic imaging apparatus according to claim 20, wherein theinformation acquisition unit is configured to acquire an indexindicating a density of the photoreceptor cells in the fundus imagebased on the periodicity.
 22. The ophthalmologic imaging apparatusaccording to claim 21, wherein the information acquisition unit isconfigured to extract a value indicating a diameter of a ring structurein a frequency image as information indicating the periodicity.
 23. Animage processing method comprising: identifying periodicity of a fundusimage obtained by capturing an image of a fundus of an eye; extracting afeature quantity based on the periodicity; and acquiring informationabout photoreceptor cells in the fundus image based on the featurequantity.
 24. A computer-readable storage medium storingcomputer-executable instructions for causing a computer to execute theimage processing method according to claim 23.